Wherefore, what is claimed is: 



1 . A computer-implemented incremental motion estimation process 
for estimating the camera pose parameters associated with each of a sequence 
of images of a scene, comprising using a computer to perform the following 
process actions: 

establishing a set of triple matching points that depict the same 
point in the scene across the three successive images of each triplet of images 
that can be formed from the sequence; 

establishing a set of dual matching points that depict the same 
point in the scene across the latter two images of each triplet of images that can 
be formed from the sequence; 

defining the camera coordinate system associated with the first 
image in the sequence as coinciding with a 3D world coordinate system for the 
scene depicted in the sequence of images; 

estimating the camera pose parameters associated with the 
second image in the sequence; and 

for each successive triplet of images formable from the sequence 
starting with the first three images, estimating the camera pose parameters 
associated with the third image of each triplet from previously-ascertained 
camera pose parameters associated with the first two images in the triplet, as 
well as the dual and triple matching points of the triplet. 

2. The process of Claim 1 wherein the process action for estimating 
the camera pose parameters associated with the third image of each triplet, 
comprises the action of computing the camera pose parameters associated with 
the third image in the triplet under consideration which will result in a minimum 
value for the sum of the squared differences between the image coordinates of 
each identified triple match point and its corresponding predicted image 
coordinates in each of the three images in the triplet, added to the sum of the 
squared differences between the image coordinates of each identified dual 



match point and its corresponding predicted image coordinates in each of the 
latter two images in the triplet. 

3. The process of Claim 2, wherein the predicted image coordinates 
5 of an either triple or dual matching point are characterized as a function of the 
3D world coordinates associated with the matching point under consideration 
and the camera pose parameters of the image containing this matching point, 
and wherein the process action of estimating the camera pose parameters 
associated with the third image of each triplet further comprises the action of 

10 computing the 3D world coordinates associated with each triple and dual match 
point in the triplet, as well as the camera pose parameters associated with the 
third image in the triplet under consideration, which will result in a minimum value 
for the sum of the squared differences between the image coordinates of each 
identified triple match point and its corresponding predicted image coordinates in 

1 5 each of the three images in the triplet, added to the sum of the squared 

differences between the image coordinates of each identified dual match point 
and its corresponding predicted image coordinates in each of the latter two 
images in the triplet. 

20 4. The process of Claim 2, wherein the predicted image coordinates 

of an either a triple or dual matching point are characterized in terms of functions 
that define the relationship between them other than their common 3D world 
coordinates. 

25 5. The process of Claim 4, wherein the functions that define the 

relationship between a triple matching point comprise: 

an epipolar relationship between pairs of the triple matching points 
in consecutive images of the triplet under consideration, said epipolar constraint 
requiring that the locations of each pair of triple matching points be related by the 

30 fundamental matrix associated with the images containing the points; and 
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a three-view relationship that defines the location of the triple 
matching point in the third image of the triplet under consideration in terms of the 
image coordinates of the triple matching points in the first two images of the 
triplet and the camera projection matrices associated with the triplet of images 
which include the camera pose parameters. 

6. The process of Claim 4, wherein the functions that define the 
relationship between a dual matching point comprise an epipolar relationship 
between dual matching points, said epipolar constraint requiring that the 
locations of each dual matching points be related by the fundamental matrix 
associated with the images containing the points. 

7. The process of Claim 1 , wherein the process action of estimating 
the camera pose parameters associated with the second image in the sequence, 
comprises the action of using a two-view structure-from-motion technique. 

8. The process of Claim 1 , wherein the process action of estimating 
the camera pose parameters associated with the second image in the sequence, 
comprises the action of applying a global bundle alignment technique to the first 
three images of the sequence of images. 

9. A local bundling adjustment system for estimating the camera 
projection matrix associated with each of a sequence of images of a scene, 
comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by 
the computing device, wherein the computing device is directed by the program 
modules of the computer program to, 

input the sequence of images, 

identify points in each image in the sequence of images; 
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determine which of the identified points depict the same 
point of the scene so as to be matching points, 

define the camera coordinate system associated with the 
first image in the sequence as coinciding with a 3D world coordinate system for 
the scene depicted in the sequence of images; 

compute the camera projection matrix associated with the 
second image in the sequence, and 

for each successive triplet of images that can be formed 
from the sequence starting with the first three images, estimate the camera 
projection matrix of the third image of each triplet from previously-ascertained 
camera projection matrices associated with the first two images in the triplet as 
well as points determined to be matching points across all three images of the 
of the triplet and points determined to be matching points across at least one 
pair of images in the triplet. 

10. The system of Claim 9, wherein the program module for 
determining which of the identified points depict the same point of the scene so 
as to be matching points comprises sub-modules for: 

establishing a set of triple matching points that depict the same 
point in the scene across the three successive images of each triplet of images 
that can be formed from the sequence; and 

establishing a set of dual matching points that depict the same 
point in the scene across the latter two images of each triplet of images that can 
be formed from the sequence. 

1 1 . The system of Claim 1 0, wherein the program module for 
estimating the camera projection matrix associated with the third image of each 
triplet comprises a sub-module for computing the camera projection matrix 
associated with the third image in the triplet under consideration which will result 
in a minimum value for the sum of the squared differences between the image 
coordinates of each identified triple match point and its corresponding predicted 



image coordinates in each of the three images in the triplet, added to the sum of 
the squared differences between the image coordinates of each identified dual 
match point and its corresponding predicted image coordinates in each of the 
latter two images in the triplet. 

12. The system of Claim 9, wherein the program module for 
determining which of the identified points depict the same point of the scene so 
as to be matching points comprises sub-modules for: 

establishing a set of triple matching points that depict the same 
point in the scene across the three successive images of each triplet of images 
that can be formed from the sequence; 

establishing a first set of dual matching points that depict the same 
point in the scene across the latter two images of each triplet of images that can 
be formed from the sequence; and 

establishing a second set of dual matching points that depict the 
same point in the scene in the first and third images of each triplet of images that 
can be formed from the sequence. 

1 3. The system of Claim 12, wherein the program module for 
estimating the camera projection matrix associated with the third image of each 
triplet comprises a sub-module for computing the camera projection matrix 
associated with the third image in the triplet under consideration which will result 
in a minimum value for the sum of the squared differences between the image 
coordinates of each identified triple match point and its corresponding predicted 
image coordinates in each of the three images in the triplet, added to both the 
sum of the squared differences between the image coordinates of each dual 
match point in the first set of dual match points and its corresponding predicted 
image coordinates in each of the latter two images in the triplet and the sum of 
the squared differences between the image coordinates of each dual match point 
in the second set of dual match points and its corresponding predicted image 
coordinates in the first and third image in the triplet. 
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14. A computer-readable medium having computer-executable 
instructions for estimating the camera pose parameters associated with each of 
a sequence of images of a scene, said computer-executable instructions 
comprising: 

identifying points in each image in the sequence of images; 
determining which of the identified points depict the same point of 
the scene so as to be matching points, 

defining the camera coordinate system associated with the first 
image in the sequence as coinciding with a 3D world coordinate system for the 
scene depicted in the sequence of images; 

computing the camera pose parameters associated with the 
second image in the sequence, and 

for each successive triplet of images that can be formed from the 
sequence starting with the first three images, estimate the camera pose 
parameters of the third image of each triplet from previously-ascertained 
camera pose parameters associated with the first two images in the triplet as 
well as points determined to be matching points across all three images of the 
of the triplet and points determined to be matching points across at least one 
pair of images in the triplet. 

1 5. A local bundle adjustment process for estimating the camera pose 
parameters associated with each of a sequence of images of a scene, 
comprising the following process actions: 

identifying points in each image in the sequence of images; 

determining which of the identified points depict the same point of 
the scene so as to be matching points, 

defining the camera coordinate system associated with the first 
image in the sequence as coinciding with a 3D world coordinate system for the 
scene depicted in the sequence of images; 



computing the camera pose parameters associated with the 
second image in the sequence, and 

for each successive triplet of images that can be formed from the 
sequence starting with the first three images, estimate the camera pose 
parameters of the third image of each triplet from previously-ascertained 
camera pose parameters associated with the first two images in the triplet as 
well as points determined to be matching points across all three images of the 
of the triplet and points determined to be matching points across at least one 
pair of images in the triplet. 

16. The process of Claim 15, wherein the instruction for determining 
which of the identified points depict the same point of the scene so as to be 
matching points comprises sub-modules for: 

establishing a set of triple matching points {(p 0 ,y , pi,y , Pzj )\j = 
1 ,. . . , M} that depict the same point in the scene across the three successive 
images of each triplet of images that can be formed from the sequence, where y 
refers to a particular instance of a triple matching point and M refers to the total 
number of triple matching points in the set; and 

establishing a set of dual matching points {(qi,/c, q** ) I k = 1,..., N} 
that depict the same point in the scene across the latter two images of each 
triplet of images that can be formed from the sequence, where k refers to a 
particular instance of a dual matching point and N refers to the total number of 
dual matching points in the set. 

17. The process of Claim 1 6, wherein the process action of estimating 
the camera pose parameters associated with the third image of each triplet 
comprises the action of computing the 3D world coordinates associated with 
each triple and dual match point in the triplet, as well as the camera pose 
parameters associated with the third image in the triplet under consideration, 
which will result in a minimum value according to the equation, 
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where / refers to the images of the triplet under consideration such that / = 0 is 
the first image , /' =1 is the second image and /' = 2 is the third image, M, 
refers to the camera pose parameters of the ; th image in the triplet, P, is the 3D 
point corresponding to triple point match (p 0 ,y, p^,j, Pzj), Qk is the 3D point 
corresponding to pair point match (qi, fe , Qzk ), <XM„ P,) represents the predicted 
image coordinates of the triple matching point py as a function of the 3D world 
coordinates associated with the triple matching point under consideration and the 
camera pose parameters of the image containing this triple matching point, and 
<KM/, Qk) represents the predicted image coordinates of dual matching point q iik 
as a function of the 3D world coordinates associated with the dual matching 
point under consideration and the camera pose parameters of the image 
containing this dual matching point. 

18. The process of Claim 17, wherein the predicted image coordinates 
0(M„ ?j) of a triple matching point p y are characterized by the equation 

p fJ =P, ?j/s, where p (J =[p^,l] 7 - p ' = A ' M " A ' is the camera internal 
parameters matrix, P, = [ [x } , Y } , Z } J l] r , and s is a non-zero scale factor, and 
wherein the predicted image coordinates 0(M„ Q/t) of a dual matching point q !ik 
are characterized by the equation q, A = P,Q k Is, where q, A = [q^,lf , and 

1 9. The process of Claim 1 5, wherein the instruction for determining 
which of the identified points depict the same point of the scene so as to be 
matching points comprises sub-modules for: 

establishing a set of triple matching points {(p 0 , 7 , pi, y , P2,j )\j = 
1 ,. . . , M} that depict the same point in the scene across the three successive 
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images of each triplet of images that can be formed from the sequence, where j 
refers to a particular instance of a triple matching point and M refers to the total 
number of triple matching points in the set; 

establishing a first set of dual matching points {(qi, fc , q 2 ,k )\k = 
5 1 , . . . , A/} that depict the same point in the scene across the latter two images of 
each triplet of images that can be formed from the sequence, where k refers to a 
particular instance of a dual matching point in the first set and N refers to the 
total number of dual matching points in the first set; and 

establishing a second set of dual matching points {(l 0 ,/ , hi ) I / = 

10 1 0} that depict the same point in the scene in the first and third images of 

each triplet of images that can be formed from the sequence, where / refers to a 
particular instance of a dual matching point in the second set and 0 refers to the 
total number of dual matching points in the second set. 

15 20. The process of Claim 1 9, wherein the process action of estimating 

the camera pose parameters associated with the third image of each triplet 
comprises the action of computing the 3D world coordinates associated with 
each triple and dual match point in the triplet, as well as the camera pose 
parameters associated with the third image in the triplet under consideration, 

20 which will result in a minimum value according to the equation, 
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min 



j=\ ,=0 k=l i=l 

O 2 



v 1=1 <e{0,2} 

where / refers to the images of the triplet under consideration such that / = 0 is 
the first image , /' =1 is the second image and / = 2 is the third image, M, 
refers to the camera pose parameters of the /' th image in the triplet, P, is the 3D 
point corresponding to triple point match (p 0 ,y, Pi,y, P2,; ), Q* is the 3D point 
corresponding to pair point match (qi, fc) q 2 ,/c ), U is the 3D point corresponding to 
the pair point match (l 0 ,/, \z,i ), <KH, P,) represents the predicted image 
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coordinates of the triple matching point p y as a function of the 3D world 
coordinates associated with the triple matching point under consideration and the 
camera pose parameters of the image containing this triple matching point, (f>(M„ 
Qi) represents the predicted image coordinates of dual matching point q;, fc as a 
function of the 3D world coordinates associated with the dual matching point 
under consideration and the camera pose parameters of the image containing 
this dual matching point, and 0(M„ U) represents the predicted image 
coordinates of dual matching point I/,/ as a function of the 3D world coordinates 
associated with the dual matching point under consideration and the camera 
pose parameters of the image containing this dual matching point. 

21 . The process of Claim 20, wherein the predicted image coordinates 
(j}{Mi, ?J) of a triple matching point p u are characterized by the equation 

p tJ = P, P 7 / * , where p, y = [p^ , if , P, = A/M/, A, is the camera internal 
parameters matrix, P, = [[^,,7, ,zjlf , and s is a non-zero scale factor, and 
wherein the predicted image coordinates (f>{M u Q k ) of a dual matching point q;,* 
are characterized by the equation q a = P,Q k /s, where q a = [q^,lf. and 
Q* =[[^*'^' Z tl 1 ] r > and wnerein tne predicted image coordinates 0(M/, L,) of a 
dual matching point I/,/ are characterized by the equation \, = P, L,/ s, where 

T^fcjf.and Z l =[[x l J ll Z,\lY' 



